Continuous Query Optimization

In large federated and shared-nothing databases, resources can exhibit widely uctuating characteristics. Assumptions made at the time a query is submitted will rarely hold throughout the duration of query processing. As a result, traditional static query optimization and execution techniques are ine ective in these environments. In this paper we introduce a query processing mechanism called an eddy, which continuously reorders operators in a query plan as it runs. We characterize the moments of symmetry during which pipelined joins can be easily reordered, and the synchronization barriers that require inputs from di erent sources to be coordinated. By combining eddies with join algorithms appropriate to large-scale environments, we merge the optimization and execution phases of query processing, allowing each tuple to have a exible ordering of the query operators. This exibility is controlled by a combination of uid dynamics and a simple learning algorithm. Our initial implementation demonstrates promising results, with eddies performing nearly as well as a static optimizer/executor in static scenarios, and adapting to run-time changes in the execution environment.